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Tangible  GIS  for  real-time  human  interaction  with  landscape  models 

L.  Mitas,  H.  Mitasova,  NCSU;  C  Ratti,  MIT 

Statement  of  the  problem 

New  mapping  technologies,  such  as  Real  Time  Kinematic  GPS,  IFSARE  or  LIDAR, 
provide  highly  efficient  means  for  3D  mapping  of  terrain  at  unprecedented  spatial  and  temporal 
resolutions.  In  response,  there  is  an  increasing  need  for  Geographic  Information  System  (GIS) 
tools  that  support  3D  modeling  and  analysis  of  topographic  surfaces  and  their  change.  Over  the 
past  decade,  powerful  3D  visualization  modules  were  developed  both  for  GIS  and  for 
specialized  applications  software  based  on  advancements  in  scientific  visualization  (Hibbard  et 
ah,  1994;  Wherrett,  2000).  While  computer  generated  models  of  topography  have  been  very 
effective  for  visual  analysis  and  communication,  interaction  with  such  models  and  their 
modification  can  be  more  challenging.  It  becomes  even  more  complex  when  the  virtual  terrain 
model  is  combined  with  results  of  sophisticated  simulations  of  landscape  processes  that  are 
important  for  prediction  of  impacts  of  terrain  changes  (Mitas  et  al.  1997).  To  fully  take 
advantage  of  these  models  for  problem  solving  and  decision  making,  effective  communication 
and  interaction  with  terrain  models  and  simulations  is  needed. 

The  complexity  of  geospatial  analysis  and  modeling  using  GIS  has  been  addressed  by 
implementation  of  graphical  user  interfaces  (Cartwright  et  al.,  2001).  On  the  one  hand,  GUIs 
with  a  high  degree  of  sophistication  have  replaced  line-driven  commands  of  first-generation 
GIS.  On  the  other  hand,  a  number  of  alternative  approaches  for  communication  with  GIS  have 
been  suggested,  most  notably  those  based  on  Virtual  Environments  (Germs  et  al.,  1999).  The 
project  focused  on  development  of  a  prototype,  novel  interface  for  GIS,  which  springs  from 
recent  work  carried  out  in  the  field  of  Tangible  User  Interfaces  (TUI;  Underkoffler  and  Ishii, 
1999)  and  is  based  on  using  a  real  3D  model  of  landscape  along  with  the  computer  generated 
virtual  model. 

Objective  of  this  short  term  innovative  research  was  to  bring  the  human-computer 
interaction  in  the  area  of  landscape  analysis,  modeling  and  design  to  a  new  level  of  effectiveness 
by  coupling  the  power  of  GIS  with  tangible  landscape  interface  and  demonstrate  a  prototype 
Tangible  GIS  (TanGIS)  environment. 

Summary  of  the  most  important  results 

The  research  focus  was  on  the  coupling  of  open  source  GRASS  GIS  (Neteler  and  Mitasova 
2004)  with  the  Illuminated  Clay  (IC)  system  (Ratti  et  al.,  2004)  with  the  goal  to  create  a 
Geospatial  Modeling  environment  that  allows  users  to  interact  with  landscape  analysis  using  a 
tactile  physical  model  of  the  studied  landscape. 

The  Illuminated  Clay  system  (Ratti  et  al.,  2004)  uses  a  commercially  available  laser  scanner 
Minolta  VIVID-900  (or  its  newer  version  VIVID  910)  to  capture  the  surface  geometry  of  the 
physical  clay  model.  The  3D  scanner  is  coupled  with  a  video  projector  that  is  used  to  project 
images  on  the  landscape  model.  The  scanner/projector  pair  is  housed  inside  a  casing 
approximately  2  m  above  the  model.  A  special  program  based  on  Minolta's  Software 
Development  Kit  (SDK)  was  developed  in  order  to  perform  one  scan  after  the  other  in  near-real 
time,  instead  of  the  standard  mode  of  operation  'scan  and  stop’.  Results  obtained  as  distance 
values  from  the  scanner's  lens  are  then  converted  into  Digital  Elevation  Models  (DEM).  Terrain 
analysis  algorithms  are  applied  to  the  DEM  and  results  are  projected  back  on  the  landscape 
model  using  a  video  projector.  The  interaction  loop  is  recursive  and  happens  in  near-real-time 
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providing  the  user  with  almost  instant  response  to  the  changes  manually  introduced  on  the 
surface.  Two  additional  projectors  (or  a  projector  and  a  computer  display)  can  be  coupled  with 
the  system  for  creating  a  virtual  terrain  model  and  the  system  interface. 

The  core  IC  software  computes  DEM  using  scanned  elevation  data,  derives  a  user- 
selected  parameter  and  re-projects  the  new  map  over  the  surface.  It  supports  computation  of 
parameters  where  only  the  elevation  data  are  required  as  inputs  -  it  can  be  therefore  considered  a 
Tangible  User  Interface  (TUI)  for  terrain  surface  analysis.  To  expand  the  use  of  the  system 
beyond  the  topography  and  include  additional  landscape  features  and  processes  we  have 
explored  coupling  of  the  system  with  open  source  GRASS  GIS.  The  tangible  landscape  model 
then  can  be  generalized  as  a  Tangible  interface  for  GIS  tasks  that  involve  change  in  terrain 
surface  and  its  attributes. 

Several  types  of  coupling  of  the  GRASS  GIS  database  and  modules  with  3D  physical 
landscape  models  were  investigated  .  The  different  types  of  coupling  are  based  on  the  type  of 
interaction  between  the  landscape  model  surface  representing  elevation,  landscape  model  color 
representing  attributes  and  a  computer  system  (GIS/IC): 

(a)  Passive  landscape  model  with  active  projection  transfers  attribute  map  from  the  computer 
to  the  physical  model.  Model  surface  is  static  and  only  a  color  map  representing  attributes  is 
projected  from  the  computer  onto  the  model.  This  one-way  coupling,  similar  to  GIS2MAP3D 
(Coucelo  et  ah,  2005),  involves  a  simple  projection  of  a  map  created  in  the  GRASS  GIS 
graphical  window  over  the  physical  model.  The  map  can  be  static,  such  as  a  combination  of 
raster  and  vector  features  (e.g.,  combined  vegetation  ,  roads,  building  footprints,  and  streams), 
or  an  animation  representing  a  landscape  process  (water  flow,  spread  of  fire).  This  approach 
offers  great  flexibility  for  the  type  of  data  that  can  be  displayed  and  additional  maps  can  be 
computed  using  GIS  modules.  Implementation  is  simple,  projecting  an  aligned  GRASS  GIS 
graphical  window  from  the  computer  screen  is  adequate;  for  larger  models  and  greater  terrain 
complexity,  image  transformation  to  central  projection  is  required. 

(b)  Active  landscape  model  with  active  projection  transfers  elevation  from  the  physical  model 
to  the  computer  and  attribute  map  from  the  computer  to  the  model.  Model  surface  can  be 
manually  modified  and  the  new  surface  elevations  are  transferred  to  the  computer  using  3D 
laser  scanning.  Attribute  data  are  projected  from  the  computer  to  the  surface  with  a  delay.  To 
support  feedback  for  terrain  modification  we  have  implemented  this  type  of  two-way 
asynchronous  coupling  of  IC  with  GRASS  GIS.  The  physical  model  is  scanned,  the  elevation 
data  are  imported  into  GRASS  GIS  and  interpolated  into  high  resolution  (1  mm)  DEM. 

Selected  analysis  is  computed  for  the  new  surface  and  the  result  is  re-projected  either  through 
the  option  (a)  or  by  importing  the  result  into  IC.  Because  the  response  does  not  happen  in  real 
time  (the  delay  depends  on  the  computational  demands  of  the  analysis),  complex  spatial 
modeling  that  involves  additional  map  layers  can  be  performed  (e.g.,  erosion  risk  modeling  that 
includes  land 

cover,  soil  and  rainfall  data  along  with  the  parameters  derived  from  the  modified,  scanned 
model).  This  approach  also  allows  us  to  derive  vector  layers  from  the  new  surface,  such  as  new 
contours  and  stream  networks,  or  perform  analysis,  modeling  or  optimization  tasks  that  may 
include  attributes  stored  in  an  external  database  system.  Complex  numerical  models 
representing  dynamic  phenomena  can  also  be  supported  with  this  option. 

(c)  Active  landscape  model  with  active  real-time  projection  transfers  elevation  from  the 
physical  model  to  the  computer  and  attribute  map  from  the  computer  to  the  model  in  real  time. 
Model  surface  can  be  modified  while  its  elevation  is  transferred  to  the  computer  using  3D 
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scanning  and  new  attribute  data  are  projected  from  the  computer  to  the  surface  in  real-time.  This 
two-way  synchronous  (real-time)  coupling  with  GRASS  GIS  that  provides  a  real-time  response 
to  manual  terrain  modifications  requires  further  research  and  development.  Several  approaches 
are  possible,  such  as  porting  selected  modules  relevant  to  terrain  analysis  from  GRASS  GIS  to 
IC  or  the  use  of  a  newly  developed  GRASS  Server  with  C  client  or  a  swig  interface. 

We  have  also  identified  additional  possibilities  for  coupling  between  the  digital  and  physical 
landscape  representation  that  can  be  developed  in  the  future: 

(d)  Active,  computer  controlled  landscape  model  with  active  real-time  projection  transfers 
the  elevation  from  the  physical  model  to  the  computer,  from  the  computer  to  the  model,  and 
attribute  map  from  the  computer  to  the  model.  This  two-way  surface  coupling  including  the 
surface  change  will  support  the  capabilities  described  by  the  option  (c),  but  it  will  also  make  the 
surface  adjustable  using  the  digital  elevation  data,  a  capability  currently  supported  by  the  pin- 
based  system.  This  approach  will  support  both  manual  and  computer  guided  modification  and  it 
will  "react"  to  the  modification  in  real  time  by  displaying  the  changes  in  selected  terrain 
parameter.  It  will  require  the  replacement  of  the  clay  surface  by  a  different  type  of  hardware. 

(e)  Active,  computer  controlled  physical  model  with  active  real-time  attribute  input  and 
projection  transfers  elevation  from  the  model  to  the  computer  and  from  the  computer  to  the 
model  as  well  as  the  attribute  map  from  the  computer  to  the  model  and  from  the  model  to  the 
computer.  This  approach  will  support  all  the  capabilities  described  in  (d)  but  it  will  also  use  the 
RGB  scan  (image)  of  the  landscape  model  to  capture  a  manually  introduced  change  in  color. 
This  will  allow  us  to  use  color  coding  of  the  surface  and  structures  to  indicate  the  type  of  land 
cover,  permeability  of  structures,  and  other  properties  needed  as  input  for  landscape  process 
modeling  and  decision  making. 

The  presented  set  of  coupling  options  will  allow  users  to  build  scalable  systems,  from  simple, 
low  cost  options  to  sophisticated  geospatial  modeling  environments  with  real-time  response  to 
different  types  of  landscape  modifications.  In  addition,  coupling  with  GRASS  GIS  provides  a 
wide  range  of  tools  needed  for  georeferenced  data  management  and  processing  that  allow  users 
to  integrate  data  from  various  sources  and  pre-process  them  into  formats  suitable  for  the 
tangible  environment.  GRASS  GIS  also  has  a  powerful  3D  visualization  module  that  further 
supports  combination  of  the  physical  model  with  an  interactive  3D  virtual  terrain  model 

Example  application  To  illustrate  the  concept  of  tangible  geospatial  modeling  environment  we 
have  used  the  coupled  IC-GRASS  GIS  system  to  explore  various  structures  and  terrain 
modifications  aimed  at  sediment  control  and  prevention  of  local  flooding  in  a  small  watershed. 
The  test  area  was  a  25  ha  site  at  the  North  Carolina  State  University  (NCSU)  Sediment  and 
Erosion  Control  Research  and  Education  Facility  (SECREF).  Various  methods  for  erosion  and 
sediment  control  are  developed  and  tested  at  this  site  and  it  has  a  sufficiently  complex  terrain  to 
demonstrate  the  functionality  of  the  tangible  interface. 

The  3D  physical  model  of  our  test  landscape  at  a  1 :2000  horizontal  scale  with  6-times 
vertical  exaggeration  (Figure  la,b)  was  based  on  a  2m  resolution  DEM  (year  1993)  interpolated 
from  0.6  m  (2  ft)  contours  obtained  from  the  Wake  county  GIS  (Figure  lc).  Airborne  lidar  scan 
of  the  area  acquired  in  year  2001  is  included  to  illustrate  actual  changes  in  topography  (Figure 
Id).  The  physical  model  is  used  as  a  tangible  interface  (Figures  2,  3)  to  explore  the  changes  in 
topography  using  various  structures,  ditches,  and  basins  to  minimize  the  flooding  and  reduce  the 
excessive  sediment  transport. 
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Figure  1.  Elevation  surface  for  the  test  site:  (a)  1:2000  scale  clay  model;  (b)  1  mm  resolution  DEM 
based  on  a  3D  scan  of  a  modified  physical  model;  (c)  original  2  m  resolution  DEM  used  to  create  the 
physical  model;  (d)  2  m  resolution  DEM  based  on  2001  year  airborne  lidar  scan.  The  surfaces  were 

visualized  in  GRASS  GIS. 

GIS-based  landscape  characterization  layers,  such  as  soils,  land  cover,  footprints  of 
structures,  and  roads  were  first  projected  onto  the  physical  model  surface  to  provide  information 
about  the  properties  of  the  modeled  landscape.  Surface  analysis,  flow  routing  and  erosion 
modeling  was  performed  in  GRASS  GIS  for  the  initial  terrain  to  assess  the  patterns  of  slope, 
overland  water  flow  (Figure  2b)  and  sediment  transport.  The  physical  landscape  model  was  then 
manually  modified  to  explore  various  approaches  for  water  flow  and  sediment  pollution  control 
(Figure  3). 

The  first  set  of  changes  involved  manual  modification  of  a  convex  hillslope  into  a  shallow 
basin  while  the  flow  direction  map  was  displayed  and  updated  along  with  the  terrain  surface 
change.  The  basin  shape  was  gradually  adjusted  until  flow  was  redirected  through  this  basin  into 
the  neighboring  forested  watershed,  thus  reducing  the  concentrated  flow  that  was  causing  the 
flooding  and  sediment  pollution.  A  new  slope  map  was  then  projected  over  the  modified  surface 
to  check  whether  the  modifications  did  not  introduce  steep  slopes  that  may  cause  stability 
problems.  Slopes  that  exceeded  the  critical  threshold,  such  as  the  red  areas  in  the  Figure  2b, 
were  manually  modified  until  the  projected  slope  map  showed  stable  values  everywhere.  The 
new  DEM  was  then  imported  into  GRASS  GIS  and  topographic  analysis  and  flow  simulation 
was  re-run  with  greater  detail.  The  process-based  hydrologic  and  sediment  transport  model 
allowed  us  to  quantify  the  impact  of  the  proposed  change  in  terms  of  water  depth  and  sediment 
flow  rates  for  a  selected  design  storm. 


Page  D-6 


Figure  2.  Clay  model  placed  in  the  IC  environment:  (a)  flow  direction  is  projected  over  the  surface  while 
additional  parameters  (elevation,  contours,  slope,  aspect,  profile)  are  displayed  by  smaller  2D  images 
around  the  worktable;  (b)  more  detailed  maps  of  water  flow  and  slopes  are  computed  for  the  scanned 
surface  using  asynchronous  coupling  with  GRASS  GIS. 


Figure  3.  Physical  landscape  model  is  modified,  with  the  impact  of  changes  shown  in  real  time  on  the 
clay  model  surface  (flow  pattern)  and  using  the  smaller  2D  images  around  the  worktable  for  elevation, 

slope,  aspect,  and  selected  profiles. 
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The  second  set  of  modifications  involved  addition  of  check-dams  in  the  main  valley.  Small, 
dam-shaped  pieces  of  clay  were  used  to  approximate  the  structures  (Figure  3).  Tactile 
environment  allowed  us  to  move  a  dam  up  and  down  along  the  valley,  add  second  and  third 
structure  and  modify  their  shape  and  size  while  observing  the  impact  on  water  flow  pattern.  The 
flow  direction  map  generated  by  IC  showed  interruption  of  flow  due  to  check-dams,  reducing 
the  flow  rate  through  the  valley  with  the  effectiveness  dependent  on  the  location  of  the  structure 
in  relation  to  the  rates  of  lateral  inflow.  The  modified  DEM  with  the  structures  was  then 
imported  into  GRASS  GIS  and  a  process  based  modeling  was  performed  to  quantify  the 
resulting  water  flow  depth  and  sediment  flow  rates  (Figure  4).  Placing  larger  structures  in  the 
lower  section  of  the  valley  resulted  in  a  more  robust  design  in  terms  of  reducing  the  flow  rates 
and  withstanding  larger  storms  than  a  set  of  smaller  structures  located  along  the  entire  valley. 
Both  two-way  synchronous  and  asynchronous  coupling  was  used  for  this  case.  In  addition,  this 
approach  allowed  us  to  investigate  the  dynamics  of  the  flow  and  asses  when  the  check-dams 
may  be  overtaken  by  storm  water. 


0  200m 

Figure  4  Water  depth  distribution  for  a  single  and  two  checkdams  -  output  from  a  numerical 

model  implemented  in  GRASS  GIS. 


Conclusion  Physical  tangible  environments  coupled  with  GIS  and  3D  visualization  have  a 
potential  to  fundamentally  change  the  user  interaction  with  digital  landscape  models  and 
georeferenced  data  representing  landscape  properties  and  processes.  For  experts,  the  problem 
solving  can  become  more  efficient  and  the  enviromnent  may  increase  the  creativity  in  finding 
potential  solutions.  In  the  area  of  decision  making,  the  system  can  make  the  digital  landscapes 
more  accessible  and  easier  to  understand  when  coupled  with  a  3D  physical  model,  potentially 
leading  to  a  more  informed  decision  making. 

Our  concept  that  includes  various  levels  of  coupling  between  the  physical  model  and  GIS  is 
flexible  and  “resource  scalable”.  It  can  be  used  for  very  simple,  low  cost  systems  that  include 
only  a  projector  and  a  landscape  model  to  sophisticated  tangible  systems  that  include  a  more 
expensive  laser  scanner  and  two-way  real  time  coupling.  The  two-way  coupling  that  will 
include  the  computer  controlled  surface  and  input  of  model  surface  color  requires  considerable 
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research  for  finding  a  low  cost,  easy  to  implement  solution,  although  there  are  already 
technologies  available  to  accomplish  this  type  of  coupling  at  a  relatively  high  cost  (Xenovision 
Mark  III).  In  future,  we  can  envision  that  large  3D  physical  models  coupled  with  GIS  and 
numerical  models,  capable  of  receiving  real-time  data  from  satellites  and  terrestrial  sensors  will 
be  at  the  heart  of  the  land  use  management  at  different  communities,  military  installations,  and 
can  one  day  improve  response  to  natural  disasters  and  emergencies  as  well  as  help  to  solve  day- 
to-day  land  management  problems.  Fort  Future  research  program  that  considers  creating 
alternative  scenarios  as  the  key  initiating  process  (Case  2002)  is  an  example  of  an  Army  program 
where  the  Tangible  GIS  enviromnent  can  fulfill  an  important  need  for  an  enviromnent  that  can 
bring  together  a  diverse  user  community  of  trainers,  operations  personnel  and  natural  resource 
managers  and  increase  the  efficiency  of  the  sustainable  installation  planning. 

Project  management  This  project  is  a  collaborative  effort  between  the  team  at  the 
Massachusetts  Institute  of  Technology,  SENSEable  City  Laboratory  lead  by  Dr.  Carlo  Ratti  and 
the  team  at  the  North  Carolina  State  University  Center  for  High  Performance  Simulation,  lead 
by  Dr.  Mitas  from  the  Department  of  Physics  and  Dr.  Mitasova  from  the  Department  of  Marine, 
Earth  and  Atmospheric  Sciences.  Dr.  Ratti  is  one  of  the  co-inventors  of  Illuminated  Clay  and  the 
NCSU  team  provides  the  expertise  in  GIS  and  geospatial  simulations.  Most  of  the  development 
and  testing  has  been  performed  using  the  Internet,  two  visits  to  MIT  allowed  the  NCSU  team  to 
work  directly  with  the  tangible  user  interfaces  using  the  NCSU  SECREF  physical  model. 

Papers  presented  at  meetings 

Mitasova  H.,  Tangible  GIS:  New  Human-Computer  Interface  For  Geospatial  Modeling,  Research 
briefing  at  USAGE  Topographic  Engineering  Center,  Sept.  2005. 

Manuscripts  accepted 

Mitasova,  H.,  Mitas,  L.,  Ratti,  C.,  Ishi,  H.  and  Hannon,  R.S.,  Real-time  Human  Interaction  With 
Landscape  Models  Using  a  Tangible  Geospatial  Modeling  Enviromnent,  accepted  to  IEEE 
Computer  Graphics  and  Applications,  Special  Issue  on  Geo  Visualization. 
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